Skip to content

Conversation

@maziyarpanahi
Copy link
Contributor

  • remove support for Apache Spark 2.3 and 2.4
  • remove Scala 2.11
  • remove unit tests for Apache Spark 2.3 and 2.4
  • make Scala 2.12 default
  • make spark 3.2 default package (spark-nlp and spark-nlp-gpu will be by default on Spark 3.2)
  • remove PySpark 2.3 and 2.4 from scripts
  • remove spark 2.3/2.4 from ResourceDownloader
  • remove unit tests for Spark 2.3 and 2.4
  • remove spark 2.3 and 2.4 from Scala and Python start() function
  • remove conditional Spark APIs when the condition was set to check Spark 2.3/2.4
  • make sure all models work on 3.0 and 3.2 (even those that were uploaded for spark 2.4 but didn't care about the Scala version)

@maziyarpanahi maziyarpanahi requested a review from danilojsl May 7, 2022 19:32
@maziyarpanahi maziyarpanahi self-assigned this May 7, 2022
@maziyarpanahi maziyarpanahi added the dependencies Pull requests that update a dependency file label May 7, 2022
@maziyarpanahi maziyarpanahi merged commit 0e84ba0 into release/400-release-candidate May 7, 2022
@maziyarpanahi maziyarpanahi mentioned this pull request May 7, 2022
8 tasks
@KshitizGIT KshitizGIT deleted the feature/remove-spark23-24-scala211 branch March 2, 2023 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants